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Abstract 

An analytical and experimental investigation 
was undertaken to model the manner in which pilots 
perceive and utilize visual, proprioceptive, and 
vestibular cues in a ground-based flight simulator. 
The study was part of a larger research effort which 
has the creation of a methodology for determining 
flight simulator fidelity requirements as its ultimate 
goal. The study utilized a closed-loop feedback 
structure of the pilot/simulator system which included 
the pilot, the cockpit inceptor, the dynamics of the 
simulated vehicle and the motion system. With the 
exception of time delays which accrued in visual 
scene production in the simulator, visual scene effects 
were not included in this study. The NASA Ames 
Vertical Motion Simulator was used in a simple, 
single-degree of freedom rotorcraft bob-up/down 
maneuver. Pilot/vehicle analysis and fuzzy-inference 
identification were employed to study the changes in 
fidelity which occurred as the characteristics of the 
motion system were varied over five configurations. 
The data from three of the five pilots that participated 
in the experimental study were analyzed in the fuzzy- 
inference identification. Results indicate that both the 
analytical pilot/vehicle analysis and the fuzzy- 
inference identification can be used to reflect changes 
in simulator fidelity for the task examined. 
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Introduction and Background 

Validating the fidelity of ground-based flight 
simulators is a problem of continuing interest to the 
simulation community. The ultimate objective of the 
research to be described is the development of a 
procedure for determining simulation requirements 
that will ensure acceptable fidelity. The need for 
such requirements can be clearly seen from Fig. 1, 
taken from Ref. 1. Here a comparison is made 
between the Cooper-Harper pilot opinion ratings 
obtained from flight test and moving-base simulator 
evaluations for a group of low-speed rotorcraft 
maneuvers. As the figure indicates, there exists a 
significant difference between the ratings given in 
flight and simulation, indicating shortcomings in the 
fidelity of the simulator. An analytical approach to 
addressing some of these problems was discussed in 
Refs. 2 and 3. 

The approach to be discussed builds upon the 
work of Refs. 2 and 3 and utilizes a model of the 
pilot’s perceptual and response characteristics as 
captured in the Structural Pilot Model discussed in 
Ref. 4. In addition, fuzzy-inference identification of 
human pilot behavior is exploited. While the use of 
fuzzy-inference models to describe human control 
behavior is not new, e.g., Refs. 5 and 6, to the 
authors’ knowledge this is the first time they have 
been employed in simulator fidelity studies. 

The Experiment 

A pilot-in-the-loop simulation of a rotorcraft 
bob-up/down maneuver using the NASA Ames 
Vertical Motion Simulator (VMS) provides the data 
base for the study. Details of this simulation can be 
found in Ref. 7. A simplified rotorcraft model was 


'Postgraduate Researcher 

2 Professor and Vice Chairman, Associate Fellow, AIAA 

Copyright ® 1999 by Y. Zeyada and R. A. Hess. Published by the American Institute of Aeronautics and 
Astronautics, Inc. with permission. 


1 



employed in the simulation study with the 
characteristics of the motion and visual systems 
serving as the experimental variables. Table 1 
describes the vehicle model and the subset of the 
motion system dynamics investigated herein. The 
task consisted of a bob-up from a stabilized hover to 
a target 32 ft from the initial rotorcraft position, a 
stabilized hover, then a bob-down to the original 
position. Figure 2 is a representation of the 
geometry of the task. Constraints were placed upon 
the translation time to ensure aggressive pilot 
behavior. Task performance standards are given in 
Table 2. Five pilots were used in the experiment and 
the data from the first three of the pilots involved in 
the simulation were analyzed here. For the purposes 
of this study, the important experimental results were 
the subjective pilot Cooper-Harper handling qualities 
ratings (HQRs), pilot-induced oscillation ratings 
(PIORs) and motion fidelity ratings (MFRs). 7 The 
scales describing these ratings are shown in Figs. 3- 
5. Table 3 summarizes the pilot rating data for all 
the three pilots whose data was analyzed in this 
study. The "Average Composite Score" shown in 
Table 3 was an attempt to obtain an average 
numerical score reflecting the HQR, PIOR and MFR 
for all three pilots values. The score was calculated 
as follows: 


Average Composite Score for Config"j" = 
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The factors 10/6 and 10 applied to the PIOR and 
1/MFR ratings means that the worst Cooper-Harper 
rating, worst PIO rating and worst fidelity rating 
would make equal contributions of 10 to the 
numerator of Eq. 1. In contrast the best ratings 
would make respective contributions of 1, 1 and 
3.33. The authors aware of the danger of mixing 
handling qualities, PIO and simulator fidelity ratings 
as suggested in Eq. 1. However, the resulting 
composite scoring approach was felt to be justified 
here given the fact that a single-axis task was 
involved, only a single experimental variable, the 
motion system dynamics, were varied and that these 
dynamics were systematically degraded from Config. 
VI in Table 1. 


Pilot/Vehicle Analysis 
Loop Structure 

Figure 6 is a block diagram representation of 
the Structural Model of the human pilot used in the 
study. This model forms the assumed pilot 
compensation in the inner-most "primary control 
loop”, i.e. that loop that forms the inner-most visual 
control loop. 2,3 The model was implemented in a 
MATLAB-based computer-aided-design packaged 
referred to as PVD NL } Figure 7 is a block diagram 
of the complete pilot/ vehicle system, with the 
structural model forming Y . Thus, an inner, 

vertical-velocity loop is assumed to serve as the 
primary control loop. The inner-loop crossover 
frequency is selected as 2.0 rad/s, as required in the 
handling qualities assessment technique of Ref. 4. 
The loop-closure sequence is based upon the fact that 
the inner, vertical-velocity loop can be closed with 
gain-like pilot compensation, and the required visual 
cue, vertical velocity can be sensed relatively easily 
by the pilot in the experimental setup. 

The outer-loop closure consists of feeding 
back vertical height error, again a variable that can 
be easily sensed by the pilot in the experiment. The 
outer loop compensation is a simple gain, i.e., 
Y p = K h . The separation between the inner and 

outer-loop crossover frequencies was selected as a 
factor of 3. Thus the outer, height loop is closed at 
0.667 rad/s. 

Modeling the Inceptor 

For the bob-up/down maneuvers that were 
the subject of this experiments and analysis, the 
cockpit inceptor was the main rotor collective 
control. The "force/feel" characteristics of the device 
are significantly different from those of most cockpit 
inceptors in that there is no self-centering 
characteristics and there is significant and deliberate 
friction introduced in the device’s dynamics, i.e. Fig. 
8. Since, in the Structural Model of Fig. 6, the 
dynamics of the inceptor are explicitly included in the 
model, it is important to at least approximate the 
dynamics of the inceptor. In the absence of a 
detailed nonlinear model of the collective, an 
approximate linear model was developed. Space does 
not permit a detailed description of the development 
of the linear representation, however on the basis of 
a computer simulation of a device with static and 
kinetic friction as shown in Fig. 8, the following 
linear model was developed: 
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chosen. 


_ 1.0 applied force q) 

Fs 5(0.15+1) displacement 

The particular units involved in Eq. 2 are not 
important here as the "gain” in the proprioceptive 
loop of the Structural Model is, in selected on the 
basis of the minimum damping ratio of any quadratic 
roots of the bJE M transfer function of Fig. 6 with 
all other loops open. 


A number of model parameters are 
considered invariant across different vehicles and 
tasks. Nominal values of these "fixed" parameters 
can be given as 

t„ = 0.2 s 

,n A, ( 5 ) 

(x> NM = 10 radjs 


Previous efforts with an earlier incarnation 
of the Structural Model have been directed toward 
modeling the pilot with a variety of inceptors, 
including those without self-centering characteristics 
like the collective control used in this study. 9 It was 
found that matching measured pilot/vehicle transfer 
functions with such inceptors required a derivative 
element to be included in the neuromuscular system 
model. This derivative element will also be included 
here. 

Choosing Pilot Model Parameters 

References 4 and 8 provide detailed 
information about choosing the Structural Model 
parameters so that the crossover model of the human 
pilot 10 is in evidence. Pilot model parameter 
selection is straightforward. Only the results are 
presented here. Elements Y NM and Y PF are given by 

2 

jjgf (3) 
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with the particular equalization of Eq. 4 dependent 
upon the form of the vehicle dynamics, Y c , around 
the crossover frequency. The zero at the origin in 
the neuromuscular system model of Eq. (3) is 
attributable to the nature of the inceptor, as has just 
been descrbied. The forms of the last of Eqs. 4 can 
be interpreted as the pilot’s "internal model" of the 
vehicle dynamics. That is, in the range of crossover, 
Y pf « 5 *T c ( 5 ) . For reasons described in Refs. 4 and 
8, a constant crossover frequency co c = 2.0 rad/s is 


C 


NM 


= 0.7 


The relatively simple relations of Eqs. 2-4 
and the crossover relation o) c = 2.0 radjs are all that 
are necessary to implement the model of Fig. 1 . The 
selection of one of the three forms on the right hand 
side of Eq. 4 is done so that the resulting open loop 
transfer function 

YY = — (/to) y (/u) * — e' V for u » « f 
P E ju> 

( 6 ) 


i.e. YYJjv) follows the dictates of the crossover 
model of the human pilot. 10 It is important to specify 
in a precise manner just how this is done. Limiting 
discussion to the last two forms of Y pF (those most 
likely to be encountered in pilot/ vehicle analyses), the 
right hand side of Eq. 4 is selected so that 


K 

Y PF <j<») 


•Y C V<*)\ 



O) = w c (7) 
arbitrary 


The gain K appearing in Eqs. 4 and 7 is chosen so 
that, with all other loops open, the minimum damping 

6 W 

ratio of any quadratic closed-loop poles of — (s) is 
r . = 0.15. Next, the vestibular loop is closed. 

» min 

This loop assumes that the time rate of change of the 
output of the primary control loop is amenable to 
sensing by the human vestibular system. The 
dynamics of a motion system such the washout 
dynamics of a flight simulator can be included in this 
loop. The gain is chosen so that with all other 
loops open but the proprioceptive loop closed, the 
minimum damping ratio of any quadratic closed-loop 
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poles of — (s) is = 0.05. Finally, K t is 

selected so that the desired crossover frequency of 
2.0 rad/s is obtained. The criteria for selecting*: 
and K* is based upon obtaining an open-loop 

M 

pilot/vehicle transfer function — (s) which exhibits, 

at least in an approximate sense, the high-frequency 
(10-15 rad/s) characteristics which have been 
measured in experiment. 10,11 


Model Overview 

The pilot model of Fig. 6 establishes the 
framework within which analytical and experimental 
results will be interpreted. This model has been 
developed over a number of years and has been used, 
in one form or another, in a variety of pilot/ vehicle 
analyses. In particular an earlier version of the 
model was used to hypothesize the manner in which 
the human pilot uses vestibular cues. 11 In the study 
of Ref. 1 1 , it was demonstrated that a model of the 
human’s use of motion cues could be presented that 
was a simple extension of that for static tracking, 
i.e., with no motion present. In terms of this model, 
motion does not provide fundamental compensation in 
tracking or regulation tasks, but rather serves to 
"tune" the pilot/vehicle dynamics by decreasing high- 
frequency phase lags occurring in the effective open- 
loop pilot/vehicle transfer function after the 
fundamental pilot compensation is accomplished 
through the proprioceptive loop closure. Figure 9 is 
a Bode plot of the pilot/vehicle transfer function Af/E 
from Fig. 6 for the nominal vehicle, showing that the 
characteristics of the crossover model of the human 
pilot are in evidence. 


Handling Qualities Sensitivity Functions 

References 4 and 8 provide detailed 
information on how the Structural Model can be used 
to provide predictions of handling qualities. The 
levels of handling qualities are predicted on the 
Cooper-Harper scale of Fig. 3. Note that, as defined 
here, level 3 handling qualities extends to a Cooper- 
Harper rating of 10. The model-based metric which 
allows prediction of HQR and PIOR levels is the 
Handling Qualities Sensitivity Function defined as 


HQSF « 



( 8 ) 


Using flight-test handling qualities results, 
Ref. 4 demonstrated that the HQSF could be used to 
discriminate between handling qualities levels 1-3. 
Figure 10 shows the HQSF bounds which resulted 
from that study. The predicted handling qualities 
level for a particular aircraft was determined by the 
area in Fig. 10 which was penetrated by the HQSF 
when the pilot model was selected as described in the 
preceding. 

Modeling Simulator Limitations 

The technique for modeling simulator 
limitations was to first complete a pilot/vehicle 
analysis of a "nominal" vehicle. The "nominal" 
vehicle represents the vehicle dynamics as 
representative of the flight article as possible, i.e., 
not including any of the flight simulator limitations, 
here limited to variations in motion system 
characteristics and to time delays associated with the 
visual and motion system operation. The pilot model 
parameters thus chosen were then frozen and the 
simulator limitations introduced. Changes in the 
HQSF were then noted and compared with the HQSF 
of the nominal vehicle. Changes in these functions 
were then considered evidence of simulation fidelity 
problems. This technique is a refinement of that 
proposed in Ref. 2 and 3. It should be emphasized 
that the resulting handling qualities predictions with 
simulator limitations are not indicative of those of the 
vehicle as simulated , since no attempt has been made 
to change the pilot model parameters from those 
obtained with the nominal vehicle model 

Inverse Dynamic Analysis 

It should be emphasized that the pilot/vehicle 
analysis to be undertaken will involve a computer 
simulation of the task using the computer-aided 
design program of Ref. 8. The pilot/vehicle system 
of Fig. 6 is clearly a compensatory one, i.e., error 
signals between command and output variables drive 
the system. While such compensatory structures are 
useful for tracking and regulation tasks, their face 
validity in the discrete bob-up/down maneuvers such 
as that being studied here is sometimes called into 
question. For example, in the past such tasks have 
been modeled by considering a precognitive rapid- 
response phase and followed by a compensatory 
error-reduction phase. 1213 . While such a modeling 
approach is useful, it complicates the analysis of the 
maneuver using the handling qualities prediction 
technique described in the preceding. Using the 
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model of Fig. 7 with a 32-ft step input command, 
however, produces unrealistic tracking results unless 
nonlinear limiters are introduced at various points in 
the pilot model. Again, this complicates the analysis, 
and requires additional assumptions about the 
saturation values of the limiters. The compensatory 
structure of Fig. 7 can be retained, however, while 
still producing realistic results by appealing to inverse 
dynamic analysis. 


The rationale behind using inverse dynamic 
analysis is that a compensatory structure can still be 
employed and produce performance comparable to 
that attained in flight or in pilot-in-the-loop 
simulation. The problem is approached by answering 
the following question: Using the compensatory 

structure of Fig. 7, what "command trajectory" h com (t) 
will produce an h(t) which approximates that obtained 
in experiment? The answer can be found through 
inverse dynamic analysis. In the examples to be 
discussed, a distinction is made between "desired" 
and "command" trajectories. The "desired" 
trajectory is the one we wish the vehicle to follow in 
the task. The "command" trajectory is one actually 
employed in the computer simulation of the 
pilot/vehicle system and which forces the closed-loop 
compensatory pilot/vehicle system to produce the 
"desired" trajectory. Here, the command trajectory 
was obtained by the expedient of approximating the 
inverse of the closed-loop pilot/vehicle transfer 
function over a frequency range extending to 
approximately 10 rad/sec, a range more than 
adequate for creating an acceptable h com Thus, the 
Laplace transform of the command trajectory 
(h com (s)) was obtained as 


h r „ m (s) = h^Xs) 


1 


h, . 

I (S) 


(9) 


J approx 


where h com (s) is the "command" trajectory as just 
defined 

h^Jis) is the "desired" trajectory as just 
defined 

l^^pprox is a proper (as many poles as 
zeros) approximation to the h/h c transfer 
function obtained from the compensatory 


pilot/vehicle system in Fig. 7, and valid up to a 
freuqency of 10 rad/s. 


In implementing Eq. 9 it was typically 
necessary to include a pure prediction time, i.e., the 
inverse of a time delay. This prediction time 
typically was called into play to approximate the 
phase lag effects of right-half plane zeros ofh/h c 
when such zeros were replaced by symmetrical left- 
half plane zeros in [hlh c l approx . Implementing such a 
predictor in simulation was accomplished by the 
expedient of eliminating the predictor and simply 
delaying h^/t) by an equal amount. 


The "desired" vehicle trajectory was defined 


here as 

KP) - 

, DISTANCE 


(' 


16 


cos(— — — ) - 9-cos(— ) + 8 

T d T d 


m 


( 10 ) 


where DISTANCE refers to the required vertical 
translational distance of the bob-up/down 
maneuvers (32 ft) 

T d is the completion time of the maneuver, 
i.e, the first time instant at which 
h^jif) = DISTANCE . Based upon the 
acceptable time-to-capture of Ref. 7, T d was 
set to 6.5 s. 

No desired trajectory such as that just created was 
included in the VMS simulation. The pilots were 
simply asked to meet the performance criteria of 
Table. 2. In the pilot/vehicle analysis, however, the 
trajectory of Eq. 10 resulted in smooth pilot model 
inputs with performance meeting the acceptable 
criteria of Table 2 with the nominal vehicle model. 

Pilot Modeling Results 
Bob-Up Maneuver Time Histories 

The pilot modeling results will be presented 
in terms of a comparison of the HQSF plots for the 
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nominal vehicle and those of the vehicle as simulated 
in the VMS experiments. Only the bob-up maneuver 
will be discussed, as the modeling results are 
essentially identical for both bob-up and bob-down. 
Figures 11-14 show the results of the bob-up 
maneuver using the program of Ref. 8. Note in Fig. 
1 1 how well the output of the compensatory 
pilot/vehicle system, h(t)> follows the h des when h com 
is applied to the pilot/vehicle system. Figure 14 is of 
particular interest as it compares the signals U M 
(proprioceptive feedback) and U s (vestibular 
feedback) in the Structural Model during the bob-up 
maneuver. Note that U s is essentially an attenuated 
version of U M . This will always be the case in the 
Structural Model since U M will always be 
proportional to the rate of change of primary control 
loop output (here h(t)) due to pilot control inputs. In 
an inanimate controller, such redundancy would be 
unnecessary. In the human controller, however, the 
vestibular signal may serve as a tuning device for the 
proprioceptive dynamics Y pf . We will have reason to 
return to the results of Fig. 14 when the fuzzy- 
inference identification results are discussed. 

HQSF Comparison 

Figures 15-19 show the HQSF for the 
nominal vehicle compared with that of each of the 
five motion configurations of Table 1 , The shaded 
areas in each figure indicate the differences between 
the HQSF for the nominal vehicle and that of the 
simulated vehicle with simulator motion limitations. 
A metric based upon these area differences was 
defined as follows and is indicated on each of Figs. 
15-19: 

Normalized HQSF Metric for Config . "/' = 

10 

/ \{HQSF(u)] r [HQSF{<») 

0 

10 

min /|[//QSF(a>)] r [//QSF(o>)] wm ^o) 

0X1 j [ o 

(ID 

The bar graph of Fig. 20 compares these areas with 


those from the Average Composite Scores of Table 3 
and a third metric yet to be described. As can be 
seen, the order of motion configurations from "most 
like" to "least like" the nominal in terms of area 
differences between HQSF’s areVl,V13,V15,V14 
and V10. This ranking is seen to agree with the pilot 
rating ranking obtained by numerically ordering the 
Average Composite Scores of Table 3. The caveat 
here is that there was no "nominal" configuration in 
the simulation study, i.e. even Config. VI contained 
time delays from the visual and motion systems. 7 
However, an estimate of the HQRs and PIORs for 
the nominal vehicle were obtained from the Structural 
Model results. These are shown in the last row of 
Table 3. 

Fuzzy-Inference Identification Results 
Introduction 

Fuzzy set theory leads to a description of 
cause and effect relationships that differ considerably 
from control theoretic approaches in describing 
dynamic systems. 14 For example, in controlling the 
height of a rotorcraft in the stabilization mode of a 
bob-up maneuver, the pilot might define his/her 
control actions as follows: 

If the height deviation is large and negative (below 
the target height) and the vertical velocity is also 
negative , my collective input is large and positive 
(i.e., commanding increased main rotor pitch), 
or 

If the height deviation is very small and negative and 
the vertical velocity is small and positive, my 
collective input is very small and negative 

The "if-then" type of statements exemplified above 
are typical of fuzzy algorithms, as are use of fuzzy- 
conditional statements such as "very small". The 
fundamental idea in fuzzy systems is a generalization 
of the concept of a set. In classical set theory, there 
is a distinct difference between elements which 
belong to a set and those which do not. Fuzzy set 
theory allows elements to belong to more than one 
set, and assigns each element a membership value, 
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A/, between 0 and 1 for each set of which it is a 
member. 

Fuzzy-inference identification (FII) can be 
thought of as a means of generating membership 
functions and if-then rules so that the inputs to a 
dynamic system can be mapped into the output(s) 
with a high degree of precision, i.e., with little 
matching error. The numerical value of the inputs 
must be "fuzzified" to allow application of FII, then 
"defuzzified" to allow mapping into the numerical 
values of the estimated output. This process has been 
automated in computer-aided design packages such as 
the MATLAB Fuzzy Logic Toolbox. 15 As will be 
described, this toolbox was utilized in analyzing the 
behavior of three of the five pilots participating in the 
NASA VMS study. The advantages of FII are that 
no particular structure need be assumed for the 
system being identified, no special inputs are 
required, and nonlinearities pose no particular 
difficulty. The primary disadvantage of FII from a 
control theoretic standpoint, is that the models to 
which feedback control engineers are most 
accustomed, e.g., transfer functions, are not natural 
products of FII. 

Interpreting Fuzzy-inference Identification Results 
- The Surface View Plot 

The if-then rules which result from FII can 
be graphically interpreted by means of surface view 
plots. Consider a system to be identified which has 
two inputs (Jtyfrj, x 2 (t)) and one output ( y(t )). Next 
set x 2 equal to its most negative value obtained in the 
identification experiment. Then allow x } to vary 
across the range of values encountered in the 
experiment and allow the if-then rules to "fire", i.e., 
allow them to create the output values. Next x 2 is 
incremented and the procedure repeated. This 
process is continued until the most positive x 2 value 
is used. A three-dimensional surface view plot can 
now be created representing y = /(;t lr r 2 ) an example 
of which is shown in Fig. 21 . If more than two input 
variables are in evidence (as will be the case herein), 
surface view plots can still be obtained, however the 


remaining input variables are held constant, typically 
at their average values obtained in the identification 
experiment. Other constant values, however, can be 
chosen. 

Fuzzy-Inference Identification and the Structural 
Pilot Model 

The analytical framework provided by the 
Structural Model can also provide some guidance in 
interpreting the results of FII through surface view 
plots. In discussing the Structural Model, it has been 
hypothesized that the activity in the proprioceptive 
feedback loop determines the handling qualities level 
assigned to the vehicle and task. This proprioceptive 
activity can also be examined in FII through surface 
view plots. Referring to Fig. 6, 6 C and h(t) serve as 
x } and x 2 with b c also serving as y. 6 C is both a 
pilot input and output and the surface view is 
indicating how the variables on the horizontal axes 
(6 c and h) are contributing to the pilot output (6 C ) 
on the vertical axis.). If there is no activity in the 
proprioceptive control loop the resulting surface view 
plot should be planar and horizontal. In addition, 
according to the fundamental hypothesis invoked Ref. 
4, the handling qualities will be optimum. This idea 
is identical to the no tracking hypothesis offered in 
Ref. 16: "Optimum handling qualities demand 

minimum closed-loop control by the pilot". As will 
be seen, one possible experimental measure of the 
activity in the primary control loop would involve the 
geometry of the surface view plot just described, i.e., 
how much it differs from a horizontal planar surface. 

Identification Results 

Figure 22 shows the assumed pilot inputs 
and pilot output for the bob-up/down maneuvers. 
Figure 23 is a sample comparison of the collective 
time history from one of the bob-up maneuvers and 
the FII which resulted. The FII output is obtained by 
using the assumed pilot inputs of Fig. 22 from the 
VMS experiment as inputs to the identified fuzzy 
model of the pilot. The training parameter 
optimization for the FII was based on a hybrid 
procedure between least squares and back 
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propagation. The FII training was performed to 
bring the root-mean squar error between model and 
pilot output to 1 TO' 5 . All the FII results were of 
this quality. As an example of the nature of the 
surface view plots, Fig. 24 shows results for a single 
bob-up and bob-down maneuver for Pilot A for 
Config. VI. 

Since three repetitions of the two separate 
maneuvers (bob-up and bob-down) were utilized in 
the experiment, some means of "averaging" these was 
necessary to obtain a single metric for comparison 
with handling qualities, etc. The following metric 
was created: 

Avg. Norm . Surface View Metric for Config. "j" 


E 

all pilots 

i-1 i=l 


mini £ 


oil j \all pilots 

[ i«l i=l 

(12) 

MJ 


where 

V BU represents the volume above minimum 
vertical coordinate on the Surface View plot 
for the "ith" bob-up 

B bd represents the volume above the 
minimum vertical coordinate on the Surface 
View plot for the "ith" bob-down 

min represents the minimum sum of the 

all j 

volumes for all configurations and pilots 

In calculating the volumes as prescribed by 
Eq. 12, care needs to be taken in interpreting 
volumes as a measure of control activity. For a case 
in which a restrained manipulator is being used, e.g., 
a self-centering control stick, the volumes in Eq. 12 
should be calculated with the "trim” manipulator 
position as the datum. The total volume above and 
below the horizontal plane defined by this datum 
value would be used in calculating the volumes in Eq. 
12. In the case of the "unrestrained" collective 


inceptor used in the VMS simulation, where no self- 
centering characteristics are in evidence, one is left 
with either using an average position over the 
maneuver as the datum, or as was done here, 
defining the datum as the most negative collective 
input. This was feasible here since, in the bob-down 
maneuver, the collective, vertical velocity and 
vertical acceleration signals were modified so that the 
surface view plots could be compared to those in the 
bob-up maneuver. For the bob-down maneuver, 
these modifications were 

( vert accel ) mod = -(vert accel) - 64 fils 1 

( 13 ) 

(collective) mod = 11 - collective in 
(vert vel ) down = -(vert vet) ft/s 

Also, the remaining input variables not included on 
the surface view plot, i.e., height error and vertical 
acceleration (for all but Config. V10) were set to 
values representing the terminal part of the 
maneuver. 

The last column of Table 3 represents the 
Average Normalized Surface View Metric (SVM) 
calculated as given in Eq. 12. The bar graph of Fig. 
20 compares the Average Normalized SVM and the 
Average Composite Scores. The order of the motion 
configurations from "most like" to "least like" the 
nominal is seen to be VI, V13, V15, V14 and V10. 
One sees that the ordering of the last two 
configurations differs from that established with the 
Average Composite Scores. Nonetheless, the results 
are noteworthy. Metrics other than Eq. 12 can be 
employed, of course. The metric of Eq. 12 was 
chosen because of its relation to activity in the 
assumed primary control loop in the bob-up/down 
maneuvers. Obviously more research in this area is 
warranted. 

Discussion 

One important hypothesis regarding pilot 
utilization of proprioceptive and vestibular cues can 
be advanced based upon the research described 


8 



herein. Namely that in a control-theoretic sense both 
the vestibular and proprioceptive feedback signals 
provide redundant information. However, the 
vestibular feedback may be of higher quality, with a 
broader bandwidth than that obtainable from 
proprioceptors. In this light, the vestibular feedback 
may serve to "tune” the dynamics of the 
proprioceptive feedback loop in addition to decreasing 
phase lags in the transfer function of the open-loop 
pilot/vehicle system. When the quality of the 
vestibular information is degraded from "nominal", 
the fuzzy-inference identification clearly shows an 
increase in proprioceptive activity which also has 
been hypothesized to be a harbinger of degraded 
handling qualities. 1 * * 4 Degradations in handling 
qualities were indeed noted in the pilot ratings from 
the simulation experiment when the motion system 
characteristics were degraded through either gain 
reduction or washout dynamics. 

The results presented have been limited to a 
single-axis control task. Extension of the results to 
multi-axis tasks is an obvious prerequisite to the 
practical evaluation of simulator fidelity. Research is 
continuing in this area. 

Conclusions 

An analytical and experimental investigation 
of the manner in which pilots perceive and utilize 
visual, proprioceptive and vestibular cues in a ground 
based flight simulator led to the development of a 
pair of metrics which were used to assess simulator 
fidelity. 

(1) The analytical approach used an established 

pilot/vehicle analysis technique based upon a 
Structural Model of the pilot from which was 
obtained Handling Qualities Sensitivity Functions. A 

metric based upon area differences between the 
Sensitivity Functions for the nominal and simulated 
pilot/vehicle system were shown to correlate well 

with composite rating scores obtained from a 
simulation experiment conducted on the NASA Ames 
Vertical Motion Simulator. 


(2) Fuzzy-inference identification was exercised 
using the NASA simulation data for three pilots. A 
total of 90 separate identification were completed. 
After identifying the "fuzzy pilot models" for each 
pilot, surface view plots were obtained which 
graphically displayed the results of the identification. 
A metric based upon the geometry of these plots was 
also shown to correlate well with the composite rating 
scores obtained from the experiment. 

(3) Further research is warranted in extending 
this methodology to multi-axis tasks, and in refining 
the fidelity metric used in the fuzzy-inference 
identification results. 
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Table 1 Vehicle Dynamics and Motion Configurations 



i(f) = Ax(t) + Bu(t) 


-0.122 

-118 

B = 

14.6 

0 

-12.9 


1 


x(t) = [h(t) z(t)] T 
u(t ) = 6 c (r-0.15) 

h(t) = vertical velocity ft/s 

z(t) - dynamic inflow parameter ft/s 2 

6 c (r) = collective displacement in 



Segment 

Desired 

Adeauate 

Ascent 

< 6 s 

< 10 s 

10 s at top 

± 2 ft 

± 5 ft 

Descent 

< 6 s 

< 10 s 

10 s at bottom 

± 2 ft 

± 5 ft 


11 



Table 3 Pilot Rating and Fuzzy Inference Identification Summary 


Confie 

HOR 

PIOR 

MFR 

Average Composite Score 

Average Normalized SVM 

Pilot 

A,B,C 

A,B,C 

A,B,C 



VI 

3,2,3 

1,1,2 

2,3,2 

1.00 

1.0 

V10 

5,4,4 

5,3,3 

1,1,1 

2.19 

2.27 

V13 

4.3,2 

4,2,2 

2,3,3 

1.21 

1.28 

V14 

5,5,4 

4,3,4 

2,2,2 

1.69 

2.39 

V15 

4,4,3 

2,2,3 

2,3,2 

1.28 

2.09 

nominal 

3,3,3 a 

2,2,2 a 

OO 00 00 
J » 

0.54 



a nominal vehicle HQRs and PIORs estimated from Structural Model results 



x MMn flight twt HQRs 

O Mean baseline 
simulation HQRs 

X Extreme ratings 


Maneuver/task 

HT = hover turn 
DQ s dash/quickstop 
BU s boto-up 
SS 3 side step 
DO 3 dolphin 



Fig. 1 A simulator fidelity problem from Ref. 1 


Fig. 2 The NASA VMS task geometry 
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Fig. 3 The Cooper-Harper rating scale 



1 NO . wi 

motions 





YES 


Cured by pitot 

1 


Sustained 

oscillations 


Divergent 


| NO 


technique 


Encountered in 
normal control 


PIO Rating 


Cured by 
sacrificing task or 
increased effort 


-*0 


Abrupt maneuvers 
only 


T-m® 


1 — ►© 


Fig. 4 The PIO rating scale 


PILOT 



Fig . 7 The hypothesized pilot/vehicle system for the 
bob-up/down maneuvers 


Motion Fidelity Scale 


Fidelity Ratine 

Definition 

Numerical Rating* 

High 

Motion sensations like 
those of flight 

3 

Medium 

Motion sensations are 
noticeably different from 
flight, but not objectionable 

2 

Low 

Motion sensations are 
noticeably different from 
flight and objectionable 

1 


* Numerical rating assigned for purposes of analysis. 



Static Friction 
Kinetic Friction 


Fig. 5 The Motion Fidelity Scale from Ref. 7 


Fig. 8 The collective inceptor 


13 














Frequency (rad/s) 

Fig. 9 The HQSF bounds for predicting handling 
qualities levels 



Frequency, rad/s 


Fig. 10 The Bode diagram of the open-loop 
pilot/vehicle system for nominal 
configuration 



Time, s 

Fig. 11 The height desired, height commanded and 
height response of the simulated pilot/vehicle 
system 



Time, s 

Fig. 12 The vertical velocity response of the 
simulated pilot/vehicle system 
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Fig. 13 The collective input in the simulated 
pilot/vehicle system 



Fig. 14 The proprioceptive and vestibular signals in 
the simulated pilot/vehicle system 



Fig. 15 The HQSF comparisons for nominal and VI 
configurations 



Fig. 16 The HQSF comparisons for nominal and 
V10 configurations 
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Frequency, rad/s 



Frequency, rad/s 


Fig. 17 The HQSF comparisons for nominal and 
V13 configurations 


Fig. 19 The HQSF comparisons for nominal and 
V15 configurations 



Frequency, rad/s 

Fig. 18 The HQSF comparisons for nominal and 
V14 configurations 


Average Average 

Composite Score Normalized SVM 

a- 1 

T 

Score/Metric Value 



Configuration 

Fig. 20 A bar graph comparison of experimental and 
analytical results 


Normalized 
HQSF Metric 



16 




y 


Fig. 21 A surface view plot 




Fig. 22 The hypothesized pilot inputs and output for 
fuzzy -inference identification in the bob- 
up/down maneuvers 



Time, s 


Fig. 23 An example of fuzzy inference 
identification: comparing collective time 

histories from experiment and identified 
fuzzy pilot model 
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Fig. 24 Surface view plots for bob-up and bob-down 
maneuver; Pilot A, Config. VI 



